138 research outputs found
A Classification Supervised Auto-Encoder Based on Predefined Evenly-Distributed Class Centroids
Classic Autoencoders and variational autoencoders are used to learn complex
data distributions, that are built on standard function approximators, such as
neural networks, which can be trained by stochastic gradient descent methods.
Especially, VAE has shown promise on a lot of complex task. In this paper, a
new autoencoder model - classification supervised autoencoder (CSAE) based on
predefined evenly-distributed class centroids (PEDCC) is proposed. To carry out
the supervised learning for autoencoder, we use PEDCC of latent variables to
train the network to ensure the maximization of inter-class distance and the
minimization of inner-class distance. Instead of learning mean/variance of
latent variables distribution and taking reparameterization of VAE, latent
variables of CSAE are directly used to classify and as input of decoder. In
addition, a new loss function is proposed to combine the loss function of
classification, the loss function of image codec error and the loss function
for enhancing subjective quality of decoded image. Based on the basic structure
of the universal autoencoder, we realized the comprehensive optimal results of
encoding, decoding and classification, and good model generalization
performance at the same time. Theoretical advantages are reflected in
experimental results.Comment: 17 pages,9 figures, 5 table
Multi-stage feature decorrelation constraints for improving CNN classification performance
For the convolutional neural network (CNN) used for pattern classification,
the training loss function is usually applied to the final output of the
network, except for some regularization constraints on the network parameters.
However, with the increasing of the number of network layers, the influence of
the loss function on the network front layers gradually decreases, and the
network parameters tend to fall into local optimization. At the same time, it
is found that the trained network has significant information redundancy at all
stages of features, which reduces the effectiveness of feature mapping at all
stages and is not conducive to the change of the subsequent parameters of the
network in the direction of optimality. Therefore, it is possible to obtain a
more optimized solution of the network and further improve the classification
accuracy of the network by designing a loss function for restraining the front
stage features and eliminating the information redundancy of the front stage
features .For CNN, this article proposes a multi-stage feature decorrelation
loss (MFD Loss), which refines effective features and eliminates information
redundancy by constraining the correlation of features at all stages.
Considering that there are many layers in CNN, through experimental comparison
and analysis, MFD Loss acts on multiple front layers of CNN, constrains the
output features of each layer and each channel, and performs supervision
training jointly with classification loss function during network training.
Compared with the single Softmax Loss supervised learning, the experiments on
several commonly used datasets on several typical CNNs prove that the
classification performance of Softmax Loss+MFD Loss is significantly better.
Meanwhile, the comparison experiments before and after the combination of MFD
Loss and some other typical loss functions verify its good universality
- …